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Abstract 


This  report  documents  the  results  of  a technical  review  of  the  BEES'*  (Building  for  Environmental 
and  Economic  Sustainability)  2.0b  software  and  user  manual.  BEES  measures  the  life-cycle 
environmental  and  economic  performance  of  building  products  using  methods  based  on  consensus 
standards.  Comments  by  four  external  experts  are  compiled  in  this  report,  along  with  responses  from 
the  BEES  developers.  While  the  life-cycle  environmental  performance  measurement  field  has 
advanced  dramatically  over  the  last  decade,  valid  disagreements  and  limitations  remain.  The  report 
sheds  light  on  the  key  issues  to  help  BEES  users  place  BEES  results  in  their  proper  context,  and  to 
help  environmental  researchers  and  tool  developers  struggling  with  the  same  concerns. 

Key  words:  Building  products,  economic  performance,  environmental  performance,  green 
buildings,  life  cycle  assessment,  life-cycle  costing,  multiattribute  decision  analysis,  sustainable 
development 


Disclaimer 

The  United  States  Department  of  Commerce  and  NIST  do  not  endorse  any  particular  brand, 
product,  or  service.  BEES  2.0  compares  generic,  U.S.  industry-average  product  classes  only  and 
no  representations  are  made  as  to  the  quality  or  fitness  of  any  specific  manufacturer's  product. 
Users  shall  not  in  any  way  say  or  imply  that  the  information  obtained  from  BEES  is  an 
endorsement  of  any  particular  product,  service,  or  brand. 

The  BEES  tool  bears  no  warranty,  neither  express  nor  implied.  NIST  does  not  assume  legal 
liability  nor  responsibility  for  a User's  utilization  of  BEES.  NO  WARRANTIES  AS  TO  ANY 
MATTER  WHATSOEVER  ARE  MADE  BY  NIST,  INCLUDING  NO  WARRANTY  OF 
MERCHANTABILITY  OR  FITNESS  FOR  A PARTICULAR  PURPOSE. 
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Preface 


This  document  is  the  result  of  a technical  review  of  the  BEES  2.0b  software  and  user  manual, 
developed  by  the  National  Institute  of  Standards  and  Technology  (NISTIR  6520).  BEES  measures 
the  life-cycle  environmental  and  economic  performance  of  building  products  using  methods  based 
on  consensus  standards.  For  environmental  performance  measurement,  the  BEES  approach  is  based 
on  the  International  Organization  for  Standardization  (ISO)  14040  series  of  life-cycle  assessment 
(LCA)  standards.  These  standards  require  a peer  review  for  LCA  studies  whose  results  are  used  to 
support  comparative  assertions.  This  document  constitutes  the  required  peer  review. 

The  review  was  conducted  at  the  request  of  the  BEES  Developer,  Ms.  Barbara  Lippiatt  of  NIST. 

Ms.  Mary  Ann  Curran  of  the  U.S.  EPA  National  Risk  Management  Research  Laboratory  served  as 
review  chair.  Comments  were  received  from  four  external  peer  reviewers  and  then  compiled  by  the 
review  chair  in  this  report.  Reviewers  were  selected  by  their  backgrounds  and  areas  of  technical 
expertise  in  order  to  form  a diverse  team  that  would  be  able  to  evaluate  all  aspects  of  the  BEES  tool. 
Short  biographical  sketches  of  the  reviewers  can  be  found  at  the  end  of  the  report.  One  of  the 
reviewers,  representing  the  industrial  sector,  asked  for  his  review  to  remain  anonymous. 

While  the  LCA  field  has  advanced  dramatically  over  the  last  decade,  valid  disagreements  and 
limitations  remain.  Indeed,  not  all  reviewers  agreed  with  all  the  issues  raised  in  this  report.  Since 
reviews  were  done  independently,  there  are  unavoidable  inconsistencies  among  some  of  the 
comments.  To  resolve  or  give  NIST  guidance  on  some  of  the  key  issues,  the  peer  review  team  held  a 
meeting  with  the  BEES  team  at  NIST  in  January  2001.  The  consensus  resulting  from  this  meeting  is 
noted  where  appropriate  throughout  the  report.  The  intent  is  to  shed  light  on  the  issues  in  an  effort  to 
help  BEES  users  place  BEES  results  in  their  proper  context,  and  to  help  LCA  researchers  and  tool 
developers  struggling  with  the  same  concerns. 

NIST  sincerely  thanks  the  entire  peer  review  team  for  generously  volunteering  their  time  and 
expertise  for  this  effort. 

Mary  Ann  Curran  and  Barbara  Lippiatt 
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1.  Introduction 


The  BEES  peer  review  team  members  were  provided  with  information  on  how  to  download  the 
BEES  software  and  user  guide  from  the  Internet.  Reviewers  did  not  inspect  primary  data  or  review 
the  TEAM  (Tool  for  Environmental  Analysis  and  Management)  model  used  to  develop  cradle-to- 
gate  data. 

Specific  review  comments  follow  and  are  grouped  by  the  following  topical  areas: 

- General  Comments 

- Installation 

- User-Friendliness 

- Format 

- Presentation  of  Results 

- Modeling  (Specifically  Impact  Modeling) 

- Data  and  Data  Sources 

- User  Support 

- User  Manual 

- Other  Recommendations 


Any  comments  that  require  a response  from  NIST  are  followed  by  comments  prepared  by  Ms. 
Lippiatt.  Her  responses  are  given  in  Italics. 
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2.  General  Comments 


BEES  2.0b  is  a very  important  tool  in  the  toolbox  of  an  environmentally-conscious  designer,  builder, 
or  product  manufacturer.  It  goes  beyond  a limited  environmental  life  cycle  assessment  to  include 
life  cycle  costs.  Such  life-cycle  based  cost  information  is  usually  lacking  in  standard  cost  calculation 
tools.  The  program  is  simple  in  its  application  and  does  not  require  information  that  is  not  readily 
available  to  the  user.  It  makes  a good  attempt  at  simplifying  a complicated  process.  If  the  user  is 
aware  of  the  limitations  and  assumptions  made  in  the  model  and  applies  this  to  their  decision 
process,  the  model  has  added  a tool  to  the  building  designer's  toolbox. 

The  BEES  tool  takes  into  account  that  the  user  may  need  to  use  highly  aggregated  information  in 
order  to  find  the  best  option.  The  capability  of  weighing  different  environmental  impacts  as  well 
as  weighing  economic  and  environmental  criteria  is  a strong  feature  of  the  tool  and  should  be 
retained  (ISO  14042  may  suggest  the  opposite,  but  this  would  “kill”  the  tool).  The  opportunity, 
as  presently  provided,  to  get  single  scores  without  aggregation  needs  to  be  maintained  for 
transparency  reasons.  Another  reviewer  commented  that  environmental  impact  categories 
should  not  be  grouped  into  a single  number,  saying  that  this  simplifies  a complicated  decision 
process  and  would  leave  the  user  with  a false  impression. 

NIST  Response:  To  accommodate  those  who  believe  that  impact  categories  should  not  be 
grouped  into  a single  environmental  performance  score,  BEES  offers  a “ no  weighting  ” option. 


The  life-cycle  assessment  (LCA)  principles  and  terminology  used  in  the  model  are  appropriate. 


The  first  step  in  any  LCA  involves  setting  the  boundaries  of  the  product  system  under  study.  In 
BEES,  system  boundaries  are  defined  by  three  decision  criteria:  mass,  energy,  and  cost 
contributions.  While  some  may  disagree  with  these  criteria,  they  do  define  the  boundaries  NIST 
wants  to  use  for  the  study. 
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3.  Installation 


Reviewers  commented  that  initially  downloading  the  software  was  easy  to  accomplish,  as  was 
installation.  One  reviewer  noted  having  problems  with  initial  installation  because  the  registration  page 
on  the  Internet  stayed  on  the  screen  after  being  submitted.  A while  later  he  realized  that  the  download 
page  appears  behind  this  page.  After  that  the  installation  worked  perfectly! 


Another  reviewer  noted  that  he  had  to  search  for  the  primary  executable  file  to  start  the  program. 
(“Maybe  I didn't  read  something  that  I should  have,  but  there  should  be  something  that  tells  me  more 
specifically  what  file  to  open  so  I don't  have  to  go  search  for  it.”) 

NIST  Response:  The  easiest  way  to  run  BEES  is  to  follow  the  installation  instructions  provided 
on  the  download  site  and  in  the  user  manual:  “ Once  installation  is  complete,  you  are  ready  to 
run  BEES  2.0  from  your  program  group  BEES.  ” These  instructions  will  be  clarified  before 
releasing  BEES  3.0  by  adding  that,  to  run  BEES,  you  simply  select 
Start  Programs  ->BEES  >BEES3  0. 


The  software  is  simple  to  use,  requiring  roughly  1 5 minutes  to  review  the  help  information  to  start 
using  the  tool  and  produce  easily  reviewable  results.  The  graphs  and  data  provided  do  give  the 
information  that  they  say  they  will  provide,  supplying  primarily  sets  of  tables  and  graphs  that  show 
you  the  results  that  you  selected  to  see  from  the  analysis  you  ran. 
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4.  User-Friendliness 


- While  BEES  is  computing  scores,  if  I try  to  switch  over  to  another  application  on  my  computer, 
the  'computing...'  window  remains  on  top  of  my  other  applications.  It  would  be  nice  if  it  would 
simply  stay  within  the  confines  of  the  BEES  window/software. 

NIST  Response:  NIST  will  fix  this  bug  before  releasing  BEES  3.0. 

- I like  that  the  “Welcome”  page  explains  the  temporal  and  spatial  applicability  of  the  data, 
particularly  stating  that  generic  product  groups  are  being  compared  within  the  tool,  not  specific 
products  from  specific  manufacturers.  The  careful  reader  will  be  well  informed  about  what  kind  of 
analysis  they  are  doing  in  BEES.  Also  about  the  “Welcome”  page,  I like  that  it  comes  up  when 
opening  BEES.  You  should  1 ) add  some  text  at  the  top  of  that  page  that  highly  recommends  the  user 
read  this  IV2-  2 pages  of  info  (fairly  short,  as  many  first  time  users  don't  want  to  do  that!),  and  2) 
make  an  option  somewhere  for  turning  off  its  opening  every  time,  after  the  user  has  read  what  he/she 
needs  to  read  [Having  an  options  area  will  certainly  allow  for  the  addition  of  more  flexibility  as  the 
tool  progresses  and  reaches  a wider  audience.] 

NIST  Response:  NIST  will  add  text  highly  recommending  reading  the  “ Welcome  " page  to  BEES 
3. 0,  and  plans  to  incorporate  option  settings  in  future  versions  of  BEES. 

- It  is  good  that  the  user  can  also  use  the  Tab  and  Alt  keys  to  move  within  the  system  (a  Windows 
OS  plus),  in  addition  to  'mousing.'  However,  in  some  interfaces  it  appears  that  the  tabbing  order 
could  be  improved. 

NIST  Response:  NIST  will  check  tabbing  order  on  all  windows  and  improve  where  needed  before 
publishing  BEES  3. 0. 

- While  the  instructions  do  inform  the  user  how  to  select  multiple  products  within  the  “Select 
Product  Alternatives”  window,  it  couldn't  hurt  to  simply  place  the  following  text  at  the  top  of  that 
window:  “Use  the  Control  key  to  select  multiple  products.” 

NIST  Response:  It  is  not  possible  to  use  the  Control  key  to  select  multiple  products , and  the 
instructions  do  not  inform  the  user  to  that  effect. 

- Why  limit  the  Zoom  In/Zoom  Out  system  to  two  levels?  It  would  seem  nice  to  allow  for  at  least  a 
few  more  considering  the  different  monitor  sizes,  resolutions,  and  visions  (eye  sight  quality)  that 
could  be  at  work  in  trying  to  review  the  results  (and  also  considering  the  limited  graphics  that  are 
supplied). 

NIST  Response:  Unfortunately,  NIST  cannot  change  the  two-level  Zoom  In/Zoom  Out  system. 

BEES  uses  a graphics  too!  to  display  its  graphs,  and  while  many  graphics  settings  may  be 
changed,  this  is  not  one  of  them.  As  more  powerful  graphics  tools  become  available,  NIST  will 
explore  using  them  in  future  versions  of  BEES. 

- The  print  feature  for  the  results  works  well. 
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- If  I am  in  the  middle  of  reviewing  some  results  (like  a graph)  and  choose  the  export  option  but 
then  choose  to  Cancel,  I don't  get  taken  back  to  the  graph,  but  get  booted  back  to  the  “Select  BEES 
Reports”  window.  Same  with  Print. 

NIST  Response:  This  is  a bug  in  the  graphics  tool  NIST  uses  that  has  been  reported  to  tool 
developers,  and  will  hopefully  be  fixed  before  NIST  releases  future  versions  of  BEES. 


- It  is  a bit  odd  to  have  execution  items  as  the  only  listings  in  the  main  menu  (when  viewing  graphs, 
“Export”  and  “Print”).  It  would  seem  advantageous  to  make  an  options  (or  the  like)  menu  choice  so 
that  exporting  and  printing  can  pull-down  off  of  this  choice.  This  would  be  more  consistent  with 
what  folks  see  in  other  applications  within  Windows.  [Not  that  Windows  is  the  best  thing  ever,  but 
it  certainly  does  appear  to  be  the  benchmark  by  which  other  systems  and  functionality  sets  are 
compared.] 

NIST  Response:  NIST  agrees.  Unfortunately,  NIST  cannot  change  the  menu  offered  by  the 
graphics  tool  it  uses.  As  more  powerful  graphics  tools  become  available,  NIST  will  explore  using 
them  in  future  versions  of  BEES 

- BEES  does  not  allow  the  user  to  select,  evaluate  and  compare  many  individual  product 
specifications  found  within  the  generic  product  classifications.  There  is  a need  to  compare  the 
environmental  performance  and  cost  of  products  with  similar  applications,  but  having  distinctly 
different  green  attributes.  As  such,  the  BEES  Program  is  not  a particularly  useful  tool  for  “fine 
tuning”  the  green  material  selections  that  are  available  within  most  generic  product  specifications. 

NIST  Response:  To  respond  to  this  need,  NIST  will  include  manufacturer-specific  product  data 
in  BEES  3.0. 
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5.  Format 


- The  BEES  PROGRAM  format  is  very  user-friendly  and  logical.  In  fact,  it  was  easy  to 
discover  most  of  the  program  features  and  reporting  possibilities  BEES  offers  even  without 
reading  the  manual.  The  multi-color,  graphical  display  makes  it  possible  to  quickly  compare 
multi-faceted  data  for  the  distinct  building  products  types,  however,  a numerical  display  is  also 
available.  The  reviewer  especially  liked  the  feature  that  allows  the  user  to  see  how  building 
products  compare  based  on  a single  attribute  environmental  impact,  such  as  global  warming,  as 
opposed  to  reporting  a score  for  combined  environmental  impacts. 
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6.  Presentation  of  Results 


- It  would  be  nice  if  the  user  knew  beforehand  that  multiple  windows  of  information  were  going  to 
open  once  he/she  selects  certain  type  of  results  to  review  and  clicks  the  “'Display”  button.  Granted, 
he/she  will  find  out  soon  enough,  but  isn't  it  the  job  of  software  designers  to  make  software  intuitive 
and  easy  to  use  and  understand?  A simple  line  of  text  at  the  top  of  the  “Select  BEES  Reports” 
window  stating  that  “multiple  windows  will  open  displaying  the  individual  results  you  have  chosen 
to  see”  should  prepare  the  user  for  what  is  to  come.  Also,  without  any  description,  I thought  the  “All 
Tables  in  One”  checkbox  was  going  to  give  me  all  the  results  in  one  pop-up  page/window  - 
incorrect. 

NIST  Response:  For  BEES  3.0,  NIST  will  cascade  multiple  results  windows  so  they  don  7 open 
entirely  on  top  of  one  another.  In  addition , to  further  explain  the  “ All  Tables  in  One  ” option, 

NIST  u'///  add  to  BEES  3.0  a detailed  description  that  will  display  when  the  mouse  moves  over  its 
checkbox. 


- As  far  as  I can  tell,  the  only  graphing  you  get  are  the  columnar  charts  that  are  shown  in  the  results. 
There  are  no  options  for  other  graphs  or  other  tabular  presentations  of  those  same  results  (flexibility 
in  presentation  limitation). 

NIST  Response:  NIST  plans  to  offer  a range  of  graph  types  in  future  versions  of  BEES. 


- The  one  graph  that  is  being  used  doesn't  really  help  any  more  than  the  scores  themselves.  It  would 
be  nice  to  see  a graph  that  allowed  me  to  see  more  of  what  the  results  are  trying  to  show... 

NIST  Response:  More  than  one  graph  is  used,  and  different  levels  of  detail  are  provided.  Don  7 
understand  this  comment. 


- It  would  seem  easier  for  me  to  see  differences  between  products  if  the  'by  life-cycle  stage'  results 
presentation  did  not  stack  up  the  life-cycle  stage  results,  but  placed  them  in  different  columns  which 
compared  each  product  side-by-side  in  the  column.  (In  the  life-cycle  stage  breakdown,  I am  not  so 
much  concerned  with  the  total  life  impacts  anymore  as  I am  trying  to  burrow  down  within  each 
life-cycle  stage.  I can  go  back  to  the  whole  life  results  if  I want  the  aggregate  breakdown.) 

NIST  Response:  NIST  plans  to  offer  a range  of  graph  types  in  future  versions  of  BEES. 


- Considering  all  the  room  left  on  most  graphic  pages,  why  not  show  the  entire  product  names 
somewhere  (like  a legend),  in  lieu  of  only  showing  a shortened  version? 

NIST  Response:  NIST  will  lengthen  product  names  on  the  graphs  before  releasing  BEES  3.0. 


- Although  the  printout  of  all  provided  result  sheets  is  very  transparent  and  includes  a lot  of  insights, 
BEES2.0b  does  not  include  a sensitivity  and  uncertainty  analysis.  This  omission  is  in  disagreement 
with  ISO  14043. 

NIST  Response:  At  present,  incorporating  uncertainty / analysis  is  problematic  due  to  lack  of 
uncertainty  > data.  For  this  reason,  at  its  Januaiy  2001  meeting,  the  BEES  Critical  Review  Team 
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advised  NIST  not  to  incorporate  uncertainty  analysis  into  BEES  in  the  short  run.  NIST  will  note 
the  disagreement  with  ISO  14043  in  its  automated  help  system  and  printed  documentation.  In  the 
long  run,  however,  one  aspect  of  uncertainty  may  he  addressed:  the  representativeness  of 
industry  averages.  That  is,  once  the  BEES  Please  program  extensively  populates  the  BEES 
database  with  manufacturer-specific  data,  the  variation  of  manufacturer-specific  data  around  its 
industry  average  data  will  become  available. 
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7.  Modeling 


Several  comments  were  submitted  regarding  the  selection  of  the  modeling  approach  as  well  as 
selection  of  the  impact  categories  that  are  used  within  the  BEES  program.  Since  these  comments 
lead  to  similar  responses,  all  the  comments  regarding  these  issues  were  grouped  together  and 
followed  by  a single  response  from  NIST. 

- Apparently,  the  impact  assessment  methodology  is  set,  and  not  changeable/flexible  (excluding 
weighting).  What  if  I wanted  to  compare  two  methodologies  across  a certain  set  of  building 
products?  As  far  as  the  user  knows,  the  already  selected  methodology  is  the  best  one.  With  the 
option  of  them  choosing  a methodology,  they  could  decide  for  themselves  (or  for  their  business) 
which  methodology  they  think  is  best. 


- Regarding  selection  of  the  LCIA  method  on  pages  8-10  of  the  user  manual,  a review  of  LCIA 
methods  is  provided  by  Dr.  Hofstetter,  an  LCIA  expert: 

The  Critical  Volumes  Approach  was  indeed  used  until  the  early  90s  but  abandoned  now 
because  (i)  fate  and  exposure  is  not  considered  (ii)  the  underlying  assumption  that  the  residual 
risk  at  thresholds  is  the  same  for  all  substances  does  not  hold  (iii)  and,  as  you  note,  legal  limit 
values  are  available  only  for  certain  chemicals  and  pollutants.  The  early  applications  of  this 
approach  go  back  to  Jansen  et  al.  (1972)  and  Basler  & Hofmann  (1974). 

- The  Ecological  Scarcity  Approach  has  been  updated  in  the  meantime  (BUWAL  1998)  and  has 
also  been  developed  for  Sweden,  Belgium,  The  Netherlands,  and  Germany.  If  the  residual  risks 
at  the  level  of  target  flows  would  be  comparable,  then  this  would  be  a revealed  preference 
approach  and  fulfill  the  requirements  set  by  SETAC.  However,  this  assumption  is  in  reality  not 
true  and  therefore  a major  point  of  criticism. 

- The  EPS  has  been  recently  updated,  see  Steen  (1999) 

- Classification/Characterization  are  the  names  that  have  been  suggested  by  SETAC  to  describe 
the  general  methodology  of  LCA.  This  description  applies  to  most  LCIA  methods  and  is 
therefore  inadequately  used.  You  may  want  refer  to  the  “Environmental  themes  approach"  or 
“Environmental  problem  approach."  Due  to  the  theoretical  and  practical  importance  of  the 
document  CML  (1992),  you  may  mention  this  reference.  An  update  of  this  method  will  be 
available  soon;  major  elements  are  published  on  the  homepage  of  CML.  Drawbacks  of  this 
method  are  the  unclear  environmental  relevance  of  the  category  indicators  that  make  the  final 
weighting  step  very  challenging/difficult. 

- Another  widely  used  method  has  been  ignored  in  this  overview,  the  Eco-indicator’99  (Goedkoop 
et  al.  1999).  The  main  disadvantages  are  the  assessment  gaps  (e.g.,  ecosystem  impact  due  to  global 
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warming)  and  its  focus  on  Europe.  Its  improvements  in  the  weighting  step  may  outweigh  these 
disadvantages. 

- Based  on  this  evaluation  it  is  not  really  clear  which  method  fits  best.  One  may  want  to  include 
even  more  than  one  method  to  test  the  sensitivity  of  the  outcome. 

NIST  Response:  The  ability  to  compare  two  impact  assessment  methodologies  is  beyond  the 
scope  o f the  BEES  pro  ject.  Rather,  BEES  assesses  impacts  based  on  the  best  available  science 
and  the  methodologies  most  accepted  in  the  United  States. 

7.1  Selection  of  Impact  Categories 

The  only  justification  for  the  selection  of  impact  categories  was  the  loose  reference  on  an 
unpublished  and  non  peer-reviewed  report  (FRED).  Neither  the  environmental  relevance  of  the 
chosen  categories  nor  the  irrelevance  of  the  non-chosen  categories  is  shown.  Udo  de  Haes  (1996) 
suggests  a list  of  impact  categories  that  may  serve  as  a starting  point. 

- The  selection  of  the  impact  assessment  method  and  the  subsequent  selection  of  impact 
categories  leads  to  problems  in  the  interpretation.  The  weighting  step  is  misleading  and  the  core 
set  of  6 impact  categories  covers  only  a subset  of  potentially  important  environmental  problems. 
The  ranking  of  product  alternatives  based  only  on  these  6 impact  categories  cannot  be 
recommended. 

- The  assessment  models  behind  the  10  impact  categories  are  in  many  cases  not  state-of-the-art 
and  need  an  update  (see  specific  comments).  Among  the  6 chosen  categories,  “solid  waste 
impacts”  and  “indoor  air  quality”  appear  in  addition  to  what  Udo  de  Haes  (1996)  or  Udo  de  Haes 
et  al.  (1999)  suggest.  “Indoor  air  quality”  is  a very  relevant  impact  for  the  evaluation  of  buildings 
and  was  therefore  a good  addition.  Another  reviewer  added  that  the  consideration  of  indoor  air 
quality  is  very  sensible  and  meaningful  since  human  beings  tend  to  spend  more  than  80  % of 
their  lifetime  in  buildings  and  since  the  dose  rates  are  higher  indoors  than  outdoors.  Further 
research  in  this  area  should  be  encouraged. 

- Solid  waste  is  usually  not  considered  to  be  an  output  of  the  product  system.  This  is  because  the 
incinerator  or  landfill  is  part  of  the  technosphere  and  only  its  effluents  and  resource  use 
(including  land  use)  are  considered  as  inventory  flows.  Therefore,  the  treatment  of  waste  has  to 
be  added  in  the  LCI. 

- If  we  believe  that  either  the  SAB  or  Harvard  weighting  scheme  is  applicable  to  BEES,  then 
this  would  show  that  the  added  4 categories  (only  included  for  driveways  and  parking  lots)  have 
a total  relevance  of  42  % and  52  %,  respectively.  This  confirms  the  correctness  of  their  inclusion 
but  also  questions  the  relevance  of  the  results  for  all  other  analyzed  elements. 

- Based  on  these  major  points,  one  has  to  assume  that  results  generated  with  the  present  version 
of  BEES2.0b  are  not  robust  against  the  mentioned  biases  and  the  ranking  results  may  be 
methodological  artifacts.  A pragmatic  way  to  improve  the  tool  would  be  the  immediate  use  of 
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state-of-the-art  impact  assessment,  and  establishment  of  a reporting  strategy  for  updated  LCI 
data  that  will  lead  to  more  credible  modeling  results  in  the  long  run. 

NIST  Response:  The  abilit)’  to  compare  two  impact  assessment  methodologies  is  beyond  the 
scope  of  the  BEES  project.  Rather,  BEES  assesses  impacts  based  on  the  best  available  science 
and  the  methodologies  most  accepted  in  the  United  States.  For  version  2.0,  the  EPA  FRED 
(Framework  for  Responsible  Environmental  Decisionmaking)  set  ofU.S.  impact  assessment 
methodologies  met  these  criteria.  There  is  general  agreement  that  the  FRED  impacts  are 
relevant.  FRED  has  since  been  eclipsed  by  the  forthcoming  EPA  TRACI  (Tool  for  the  Reduction 
of  Chemical  and  Other  Impacts)  methodology > set,  which  will  be  integrated  into  BEES  3.0.  At  its 
January  2001  meeting,  the  BEES  Critical  Review  team  agreed  with  this  approach. 

NIST  does  not  claim  that  non-chosen  impacts  are  irrelevant.  Rather,  the  BEES  2. 0 
documentation  and  online  help  system  discuss  the  two  categories  for  which  valid  U.S.  impact 
assessment  methodologies  did  not  exist — land  and  water  use — and  gives  guidance  for 
interpreting  BEES  results  in  this  context.  The  new  TRACI  land  use  impact  assessment 
methodology > is  being  incorporated  into  BEES  3. 0. 

At  its  January ’ 2001  meeting,  the  BEES  Critical  Review  Team  agreed  that  modeling  the  fate  of 
solid  waste  is  an  important  topic  for  long-term  research.  Until  research  yields  the  understanding 
necessary > to  develop  a more  refined  assessment  methodology’  for  this  impact,  however,  the 
volume  of  solid  waste  to  the  landfill  is  a satisfactory > proxy  for  evaluating  the  solid  waste  impact. 
Almost  95  % ofU.S.  construction  and  demolition  waste  is  currently  landfilled.  Further,  the 
“direct  use  of  inventories  ” approach  BEES  currently  takes  for  solid  waste  is  similar  to  that 
taken  by  TRACI  for  water  use,  and  for  similar  reasons. 


The  science  and  data  underlying  measurement  of  the  added  four  impacts  are  not  as  agreed  upon 
and  available  as  that  for  the  original  six  impacts,  and  the  added  four  impact  measures  (which 
are  also  available  for  two  floor  coverings)  are  quali  fied  accordingly  in  BEES.  Indeed,  new 
science  will  be  used  to  assess  most  of  these  added  impacts  in  the  upcoming  BEES  3.0.  The  BEES 
2.0  documentation  and  online  help  system  give  guidance  for  interpreting  BEES  results  based  on 
an  incomplete  set  of  impacts.  It  is  better  to  scienti  fically  assess  an  incomplete  set  of  impacts 
rather  than  no  impacts  at  all,  thereby  allowing  users  to  focus  their  judgment  on  missing  impacts 
rather  than  making  everything  a matter  of  judgment.  The  issue  of  updating  BEES  data  to  enable 
assessment  of  all  10  impacts  for  all  products  was  raised  at  the  January  2001  meeting  of  the 
BEES  Critical  Review  Team.  Since  national  standard,  “backbone  ” life-cycle  inventory  data  will 
likely  be  developed  over  the  next  several  years,  it  was  agreed  that  NIST  should  hold  off 
expanding  its  existing  data  to  cover  all  1 0 impacts  until  these  data  become  available. 


7.2  Characterization  Models 

- Global  Warming:  The  integration  time  horizon  is  neither  mentioned  nor  justified.  The 
exclusion  of  a large  number  of  further  greenhouse  gases  was  not  mentioned  at  all  (especially, 
CFCs,  H-CFCs,  N20). 

NIST  response:  In  BEES  3.0,  NIST  will  mention  and  justify  the  100-year  time  horizon  it  uses,  as 
is  common  LCA  practice.  Life-cycle  inventory  data  collected  for  BEES  2.0  include  the  additional 
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greenhouse  gases  cited , but  for  consistency  reasons , /VAST  chose  not  to  include  these  gases  in  its 
global  warming  assessment  until  it  could  update  BEES  1.0  inventory  data  to  include  them  as 
well.  The  issue  of  updating  BEES  1.0  data  to  enable  assessment  of  additional  greenhouse  gases 
was  raised  at  the  January  2001  meeting  of  the  BEES  Critical  Review  Team.  Since  national 
standard,  “ backbone  ” life-cycle  inventory  data  will  likely  be  developed  over  the  next  several 
years,  it  was  agreed  that  NIST  should  hold  o ff  updating  its  BEES  1.0  data  to  cover  these  gases 
until  these  backbone  data  become  available.  However,  it  is  believed  that  the  additional  gases 
will  not  significantly  affect  the  global  warming  scores.  The  most  important  flow  continues  to  be 
carbon  dioxide.  NIST  will  document  the  above  issues  in  BEES  3. 0. 


- Acidification:  Does  the  word  “hydrogen"  imply  that  it  is  a H+  ? 

NIST  Response:  Yes.  NIST  will  clarify  that  these  are  hydrogen  ions  in  BEES  3.0. 

- Nutrification:  Is  the  value  for  ammonia  correct?  Own  calculation  suggests  0.35. 

NIST  Response:  Yes,  the  value  for  ammonia  in  water,  0.42  is  correct.  Note,  however,  that  the 
TRACI  eutrophication  impact  assessment  methodology > is  being  incorporated  into  BEES  3.0,  so 
this  value  will  change. 

- Natural  Resource  Depletion:  The  reasoning  for  including  resource  depletion  does  not  support 
the  choice  of  the  model.  While  it  is  argued  that  this  is  a long-term  sustainability  issue,  it  is 
calculated  with  economically  and  sub-economically  extractable  resources  rather  than  a more 
future-oriented  measure  (see  Guinee  et  al.  1995,  Mueller- Wenk  1998,  Goedkoop  et  al.  1999,  and 
Steen  1999).  It  is  also  unclear  why  biotic  resources  including  land  use  are  excluded. 

N 1ST  Response:  In  BEES  3.0,  NIST  will  clarify  the  discrepancy  between  selection  of  impact  and 
choice  of  model.  A valid  assessment  technique  for  land  use  was  not  available  for  inclusion  in 
BEES  2.0.  The  TRACI  land  use  impact  assessment  methodology / is  being  incorporated  into  BEES 
3.0. 

- Indoor  Air  quality:  It  is  excellent  that  this  category  is  added,  and  due  to  the  lacking  state-of- 
the-art  method,  one  can  understand  the  particular  problems.  However,  there  are  some  potential 
improvements  that  may  be  considered:  (i)  the  unit  in  Table  2.6  should  read  mg/rnYh  rather  than 
mega-gram,  (ii)  the  chosen  key  to  add  emissions  from  the  floor  coverings  and  the  installation 
adhesives  is  unnecessarily  wrong.  The  chosen  key  implies  that  1.667  mg/m  /h  at  24  h is  the  same 
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as  a total  release  of  3.2732  kg/m".  If  this  were  true  and  if  we  make  the  unrealistic  assumption  that 
the  emission  rate  at  24  h stays  stable,  then  the  floor  covering  has  to  last  224  years  before  the  two 
emissions  are  indeed  equal.  Instead,  it  is  suggested  to  integrate  an  exponential  function  that 
shows  after  24  h the  given  rate  and  approaches  a negligible  rate  after  a couple  of  years.  This 
integrated  amount  can  then  be  added  to  the  installation  adhesives  (and  will  be  in  general  much 
smaller).  It  is  also  not  clear  why  it  was  necessary  to  add  all  different  VOCs  1:1.  Later,  in  the 
humantox  paragraph,  a very  sophisticated  method  is  chosen  to  account  for  a large  number  of 
VOCs.  The  toxicological  potencies  used  there  (without  fate,  only  for  inhalation)  could  also  be 
used  for  the  indoor  situation.  What  is  the  source  for  coat  reapplication  of  every  4 years  over  50 
years?  (This  is  much  more  frequent  than  what  I experience  and  would  equal  1 2 layers,  which 
makes  it  necessary  to  scrub  the  walls  in-between.  This  scrubbing  is  not  included.) 
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NIST  Response:  For  BEES  3.0,  NIST  will  turn  off  its  automatic  (and  erroneous  in  this  case) 
capitalization  setting  within  MS  Word  to  correct  the  error  cited  in  item  i.  Emissions  from  floor 
coverings  and  adhesives  are  not  added  together.  Rather,  because  they  are  expressed  in  different 
units,  they  are  each  first  normalized  by  dividing  by  the  corresponding  emissions  for  the  worst- 
performing product,  then  the  normalized  values  are  averaged.  Different  VOCs  are  added  1:1  for 
data  availability > reasons,  not  for  assessment  technique  availability > reasons.  The  source  for  coat 
reapplication  every  4 years  is  Whitestone  Building  Maintenance  and  Repair  Cost  Reference  and 
reflects  commercial  building  practice.  In  future  versions  of  BEES,  NIST  will  address  the 
scrubbing  issue. 

- Smog  formation:  The  present  text  does  not  provide  a MIR  for  NOx.  What  is  the  source  for 
footnote  33?  How  do  MIRs  compare  with  the  POCPs  used  by  OECD  and  introduced  in  CML 
(1992)? 

NIST  Response:  For  BEES  3.0,  NIST  will  be  updating  its  smog  assessment  by  replacing  the  EPA 
FRED  approach  with  the  forthcoming  TRACI  approach.  At  the  time  of  BEES  2.0  publication, 
FRED  offered  the  best  available  science.  FRED  has  since  been  eclipsed  by  the  forthcoming  EPA 
TRACI  approach,  so  NIST  will  be  using  the  TRACI  smog  assessment  technique,  which  includes 
an  MIR  for  NOx,  in  BEES  3.0.  The  source  for  footnote  33  is  the  FRED  report. 


- Ecological  toxicity:  From  the  given  description  the  source  for  the  method  is  unclear.  Based  on 
the  provided  information  and  own  interpretation  the  following  comments  can  be  made: 

1 . Step  1 makes  sure  that  only  clear  differences  between  emissions  are  included.  This  step 
can  be  criticized.  First,  the  15  % minimum  spread  is  only  meaningful  if  the  measurement  errors 
are  larger  than  that.  Second,  the  effect  of  this  procedure  is  that  the  relative  ecotoxic  difference 
between  alternatives  will  increase.  Within  the  chosen  weighting  scheme  this  is  entering  a bias  to 
“overweight' ' ecotox  impacts. 

2.  In  step  2,  it  is  unclear  which  benchmarks  have  actually  been  chosen.  LCA  deals  with 
optimization  beyond  compliance.  Therefore,  acute  effects  would  not  occur. 

3.  Steps  4 and  5 will  lead  to  a compression  of  the  toxicity  factors  towards  the  upper  end, 
which  makes  relative  differences  between  substances  smaller  than  they  are. 

4.  Steps  6-8:  In  the  meantime,  multi-media  models  have  been  developed  that  may  produce 
better  fate  and  exposure  information  than  the  suggested  approach.  This  is  especially  true 
because  those  multi-media  results  have  been  used  for  the  human  toxicity  potentials. 

This  first  analysis  suggests  that  the  chosen  approach  may  be  flawed  and  enter  biases.  The  most 
obvious  step  would  be  to  combine  the  fate  and  exposure  information  available  from  Hertwich 
(1999)  - the  humantox  model  for  TRACI  - with  the  chronic  toxicity  factors  suggested  by  RTI. 
Considering  the  fact  that  the  present  approach  does  not  include  any  U.S. -specific  data,  one  may 
also  use  the  data  provided  by  Huijbregts  (2000)  before  TRACI  (the  U.S. EPA  LCIA  method) 
provides  American  factors. 

NIST  Response:  The  source  for  the  ecological  toxicity  > method  is  the  EPA  FRED  report.  NIST 
will  be  updating  its  ecological  toxicity ; approach  by  replacing  the  FRED  approach  with  the 
forthcoming  TRACI  approach  in  BEES  3.0.  At  the  time  of  BEES  2.0  publication,  FRED  offered 
the  best  available  science. 
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- Human  toxicity:  This  paragraph  refers  to  an  unpublished  secondary  source.  It  is  suggested  to 
refer  directly  to  the  latest  version  of  the  model  (Hertwich  1999).  This  approach  does  not  follow 
the  1LSI  recommendations  because  no  weighting  between  different  effects  was  included.  The 
suggested  equal  weighting  of  carcinogenic  and  non-carcinogenic  effects  is  unclear.  Does  this 
mean  that  Benzene  equivalents  are  added  with  Toluene  equivalents?  If  this  is  the  case,  then  it 
implies  that  those  two  substances  are  equally  hazardous.  A more  reasonable  assumption  would 
be  that  the  used  benchmark  doses  for  the  two  metrics  have  a similar  meaning  which  allows  one 
to  add  the  non-normalized  scores. 

NIST  Response:  In  BEES  3.0,  NIST  will  refer  directly  to  Hertwich  1999.  No,  benzene  and  toluene 
equivalents  are  not  added.  Rather,  each  is  first  normalized  by  dividing  by  the  corresponding 
equivalents  for  the  worst-performing  product,  then  a simple  average  of  the  normalized  values  is 
taken.  NIST  will  clarify > this  approach  in  the  BEES  3.0  documentation. 

- The  effects  of  criteria  air  pollutants  on  human  health  are  only  partly  covered.  In  particular,  the 
large  impact  of  fine  particles  - which  may  outweigh  the  impacts  of  all  other  pollutants  that 
contribute  to  human  toxicity  - is  fully  omitted.  It  may  be  most  efficient  to  work  together  with 
Jane  Bare,  U.S.EPA,  to  implement  TRACI  (Tool  for  the  Reduction  of  Chemical  and  Other 
Impacts)  in  BEES  as  soon  it  becomes  available.  TRACI  is  supposed  to  deal  with  these  problems. 
NIST  Response:  Yes,  NIST  is  currently  working  with  Jane  Bare  to  implement  TRACI  in  BEES 
3.0.  At  the  time  of  BEES  2.0  publication,  FRED  offered  the  best  available  science. 

- The  offered  transport  distances  from  the  manufacturer  to  the  user  are,  in  the  example  of  wall 
insulation  (I  checked  only  this),  probably  too  short.  This  would  imply  that  you  have  a large 
number  of  manufacturers. 

NIST  Response:  NIST  originally  had  longer  travel  distances  for  insulation.  It  shortened  them 
based  on  comments  received  from  the  insulation  industry  during  its  Beta  review  of  BEES  1.0. 
Insulation  is  such  a voluminous  product  that  long  travel  distances  are  not  cost-effective,  and  thus 
not  common  industry  practice. 

- The  user  manual  does  not  document  energy  and  transportation  processes  - the  backbones  of 
most  LCAs.  When  you  refer  to  energy  use  you  do  not  specify  the  assumed  technology.  The  type 
of  burners  assumed  should  be  relevant  as  well. 

NIST  Response:  Reliable  U.S.  backbone  data  are,  at  present,  only  available  from  commercial 
tools.  BEES  does  not  publish  transparent  backbone  data  because  none  are  publicly  available 
and  BEES  is  not  intended  to  compete  with  private  sector  tools.  Rather,  NIST  contracts  with 
Environmental  Strategies  and  Solutions  and  PricewaterhouseCoopers/Ecobalance,  a world-class 
LCI  tool  developer,  for  aggregated  backbone  data,  thus  protecting  their  intellectual  property ;. 
However,  publicly  available  U.S.  backbone  data  will  likely  be  developed  over  the  next  several 
years.  Once  available,  NIST  will  incorporate  these  more  transparent  data  into  future  versions  of 
BEES. 

- Functional  unit:  The  definition  provided  on  page  6 suggests  that  the  compared  alternatives  are 
“true  substitutes.”  However,  only  footnote  104  gives  an  idea  what  “true”  means.  Due  to  the 
importance  of  this  point  it  is  suggested  that  a table  be  added  where  for  each  individual  element 
the  “true”  is  specified,  i.e.,  which  additional  characteristics  next  to  surface  and  lifetime  have 
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been  adjusted  to  correct  for  different  properties  and  which  characteristics  may  still  differ  and  be 
relevant  for  the  interpretation. 

NIST  Response:  NIST  will  odd  the  suggested  table  in  BEES  3.0. 

- It  should  be  mentioned  that  occupational  health  was  excluded  from  the  present  analysis. 

NIST  Response:  NIST  will  mention  the  exclusion  of  occupational  health  in  BEES  3.0. 

- Due  to  the  lack  of  uncertainty  and  sensitivity  analysis  an  interpretation  in  accordance  with  ISO 
14043  is  not  possible. 

NIST  Response:  This  issue  was  discussed  at  the  January  2001  meeting  of  the  BEES  Critical 
Review  Team.  At  present , incorporating  uncertainty > analysis  is  problematic  due  to  lack  of 
uncertainty > data.  For  this  reason,  the  BEES  Critical  Review  Team  advised  NIST  not  to 
incorporate  uncertainty > analysis  into  BEES  in  the  short  run.  NIST  will  note  the  disagreement 
with  ISO  14043  in  its  automated  help  system  and  printed  documentation.  In  the  long  run, 
however,  one  aspect  of  uncertainty > may  be  addressed:  the  representativeness  of  industry 
averages.  That  is,  once  the  BEES  Please  program  extensively  populates  the  BEES  database  with 
manufacturer-speci  fic  data,  the  variation  of  manufacturer-speci  fic  data  around  its  industry 
average  data  will  become  available. 

- The  following  statement  was  noted  on  page  15  of  the  BEES  2.0  user  manual: 

“Equivalency  factors  have  not  been  developed  to  consider  the  ultimate  fate  of  the  non-recyclable 
solid  waste  (e.g.,  landfill  leachate,  gas  or  incinerator  emissions,  and  ash).  Thus,  the  Direct  Use 
of  Inventories  Approach,  described  at  the  beginning  of  this  subsection,  is  used,  with  solid  waste 
volume  representing  the  solid  waste  impact  of  the  product.” 

Of  course,  a measure  of  solid  waste  generation  (volume)  alone  is  not  a meaningful  indicator  of 
the  solid  waste's  potential  to  do  environmental  damage.  In  particular,  an  inert  waste,  such  as 
concrete,  is  much  less  likely  to  create  measurable  environmental  problems  (leachate,  landfill  gas, 
ash)  as  compared  to  organic  waste  and  metals  that  tend  to  undergo  complex  chemical  reactions 
under  aerobic,  anaerobic,  and  acidic  landfill  conditions.  This  model  would  tend  to  either  over- 
estimate or  under-estimate  the  importance  of  the  waste,  depending  on  the  type  of  waste. 

Some  “solid  waste  equivalency  factors”  might  be  derived  from  existing  research.  For  example, 
there  are  probably  studies  that  estimate  ash/landfill  gas/leachate  production  by  different  fractions 
of  the  waste  stream  based  on  chemical  stoichiometry.  In  particular,  there  are  numerous  studies 
available  on  methane  generation  from  solid  waste.  The  Los  Angeles  County  Sanitation  District 
and  the  University  of  Wisconsin  are  reportedly  at  the  forefront  of  this  research. 

Other  important  indicators  of  solid  waste's  impact  on  the  environment  include  the  percentage  of 
landfills  meeting  federally-mandated  minimum  standards  for  landfill  design  and  operation 
(including  lining  requirements  and  leachate  and  gas  recovery),  the  existence  of  state  and  local 
recycling  laws  for  various  construction  waste  constituents,  average  rainfall,  and  state  recycling 
rates  for  specific  solid  waste  fractions.  Of  course,  many  “environmental  indicators”  for  solid 
waste  would  tend  to  vary  by  region.  As  such,  BEES  programming  may  be  improved  by 
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incorporating  geographical  variables  in  calculating  various  environmental  assessments.  It  is 
noted  that  the  current  goal  of  BEES  is  to  establish  a scope  based  on  U.S. -average  results. 

NIST  Response:  At  its  January  2001  meeting,  the  BEES  Critical  Review  Team  agreed  that 
modeling  the  fate  of  solid  waste  is  an  important  topic  for  long-term  research.  Until  research 
yields  the  understanding  necessaiy  to  develop  a more  re  fined  assessment  methodology’  for  this 
impact,  however,  the  volume  of  solid  waste  to  the  landfill  is  considered  by  the  BEES  developers 
to  be  a satisfactoiy  proxy  for  evaluating  the  solid  waste  impact.  Almost  95  % of  U.S. 
construction  and  demolition  waste  is  currently  landfilled.  Further,  the  “direct  use  of 
inventories  ” approach  BEES  currently  takes  for  solid  waste  is  similar  to  that  taken  by  TRACI  for 
water  use,  and  for  similar  reasons. 


- The  BEES  Program  allows  the  user  to  set  a transportation  distance  parameter  to  account  for 
shipping  finished  commercial  carpet  from  the  manufacturing  plant  to  the  project  site.  However, 
this  feature  does  not  appear  to  have  range  setting  that  is  appropriate  for  the  West  Coast.  While 
the  highest  transportation  setting  is  only  1000  miles,  most  commercial  nylon  carpet  shipped  to 
California  originates  in  Georgia — a significantly  longer  distance  than  the  1,000-mile  setting  in 
the  BEES  program.  It  is  unclear  to  the  reviewer  whether  or  not  this  longer  transportation 
distance  is  accounted  for  in  BEES.  It  may  prove  necessary  to  program  a wider  range  of  distances 
for  the  nylon  commercial  carpet  selections  to  account  for  the  significant  transportation  impact  of 
shipping  this  product  to  the  West  Coast. 

NIST  Response:  For  BEES  3.0,  NIST  is  checking  its  assumed  transportation  distances  with  the 
carpet  industry  and  will  adjust  them  if  necessaiy. 

- Early  on  in  LCA  development  it  was  suggested  that  organic  compounds  not  be  grouped 
together  as  a single  inventory  item.  BEES  still  aggregates  organic  compounds  as  one  group. 
Potential  environmental  impact  will  vary  depending  on  the  compound. 

NIST  Response:  This  limitation  is  documented  in  the  BEES  manual  (pp.  18-20).  Total  VOCs  are 
used  to  assess  indoor  air  cpiality’  because  data  permitting  speciation  of  individual  organic 
compounds  are  not  available.  NIST  is  encouraging  manufacturers  to  begin  providing  speciated 
data. 

- The  flows  for  a product  are  reduced  to  those  that  have  a significant  contribution  to  potential 
impacts.  Who  made  this  decision?  Was  an  environmental  background  required? 

NIST  Response:  Yes,  the  NIST  LCI  data  contractors,  Environmental  Strategies  & Solutions  and 
PricewaterhouseCoopers/Ecobalance,  have  world-class  environmental  backgrounds  and 
experience.  Moreover,  flows  having  a significant  contribution  to  potential  impacts  are  in  fact 
prede  fined  by  the  state-of-the-art  impact  assessment  methods  BEES  uses.  For  all  BEES  products, 
flows  identified  by  these  methods  are  collected.  NIST  will  clarify  its  BEES  3.0  documentation  to 
this  effect. 


- The  BEES  model  only  considers  ten  environmental  impacts.  There  are  other  impacts  that  could 
be  considered  in  the  process.  EPA  has  evaluated  six  of  the  impacts  used  in  the  BEES  model  and 
verified  their  scientific  validity  (Global  Warming  Potential,  Acidification  Potential, 
Eutrophication  Potential,  Natural  Resource  Depletion  and  Indoor  Air  Quality).  While  EPA  has 
reviewed  these  six,  two  are  still  somewhat  controversial:  Natural  Resource  Depletion  and  Indoor 
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Air  Quality.  The  reader  needs  to  be  aware  of  this  disagreement.  Also,  Ecological  Toxicity  and 
Human  Toxicity  still  have  controversial  assessment  methodologies.  The  main  users  of  BEES 
will  have  a limited  background  in  Environmental  Science  and  the  BEES  Model  simplifies  a 
complicated  process.  The  user  needs  to  be  made  aware  that  some  impact  models  are  still  in  the 
development  stages. 

NIST  Response:  At  its  January  2001  meeting , the  BEES  Critical  Review  Team  agreed  that  BEES 
should  assess  the  impacts  for  which  EPA ’s  TRACI  is  developing  state-of-the-art  U.S. 
methodologies.  These  impacts  include  8 of  the  10  BEES  environmental  impacts.  Further,  the 
BEES  Critical  Review  Team  endorsed  the  inclusion  of  indoor  air  quality’.  See  discussion  above 
regarding  the  inclusion  of  the  solid  waste  impact.  Assessment  techniques  have  evolved  such  that 
the  earlier  controversies  are  now  less  serious,  and  while  techniques  will  continue  to  evolve  for 
some  time,  they  are  well  beyond  the  development  stages.  For  BEES  3.0,  NIST  will  update  its 
Limitations  documentation  to  inform  the  user  of  these  issues.  See  discussion  above  regarding 
other  impacts,  primarily  land  use,  not  included  in  BEES  2. 0. 


7.3  Normalization  and  Weighting 

There  was  disagreement  among  the  reviewers  regarding  the  application  of  weighting  to  the 
impact  categories: 

- The  weighting  system,  including  the  user-definable  portion,  works  very  nicely.  And  the 
checkbox  for  “No  weighting-"  is  well  executed. 

- Depending  on  the  location,  some  environmental  categories  would  carry  more  weight  than 
others.  While  the  Multiattribute  Decision  Analysis  (MADA)  is  an  accepted  methodology,  it 
should  not  be  applied  here. 

NIST  Response:  NIST  does  not  understand  the  conflict  between  the  location-specific  issue  and 
the  applicability  of  MADA.  Indeed,  the  ability ? to  change  weights  based  on  location  is  a strength 
o f BEES. 

- The  weighting  step  is  not  congruent  with  the  modeled  category  indicators.  The  resulting 
weighted  scores  are  therefore  arbitrary  and  shall  not  be  used  for  ranking  product  alternatives. 
Major  Response  # 1 to  MADA  Critique 

NIST  Response:  While  NIST  agrees  that  applying  the  AST M standard  MADA  approach  is  a 
second-best  solution,  it  strongly  maintains  that  at  the  time  of  BEES  2. 0 publication  it  was  the 
most  appropriate  approach,  given  the  lack  of  U.S.  normalization  data.  Indeed,  MADA  is 
intended  as  a technique  for  application  in  situations  in  which  normalization  data  are  not 
available.  Further,  the  MADA  limitations  are  clearly  discussed  in  the  BEES  2.0  online  and 
printed  Limitations  documentation.  At  a meeting  between  BEES  and  TRACI  developers  following 
the  January  Critical  Review  Team  meeting,  EPA  noted  that  U.S.  normalization  data  would  be 
available  for  BEES  3.0.  This  will  permit  BEES  to  evolve  from  a relative  to  an  absolute 
measurement  system,  and  will  be  an  important  advance  that  will  be  incorporated  into  BEES  3. 0. 

- According  to  Keeney  et  al.  (1976),  multi-attribute  utility  analysis  requires  utility 
independence.  This  condition  is  not  strictly  fulfilled,  as  scores  for  humantox  and  ecotox  and  for 
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acidification  and  eutrophication  tend  to  be  highly  con-elated.  Therefore,  the  statement  on  page 
28  that  MADA  is  the  most  appropriate  technique  is  not  so  clear.  However,  this  assumption  may 
be  considered  to  be  state-of-the-art.  As  a first  step  an  internal  normalization  is  suggested. 

Indeed,  internal  normalization  is  one  of  the  scaling  techniques  used  in  MAUT.  However,  its 
application  in  the  LCA  context  violates  the  congruence  required  between  the  weighting  step  and 
the  category  indicators. 

Major  Response  # 2 to  MADA  Critique 

NIST  Response:  MADA  does  not  belong  to  the  family  of  multi-attribute  utility  theory  (MAUT) 
approaches.  Rather , MADA  is  a more  pragmatic  approach  than  the  theoretically  appealing,  yet 
impractical,  MAUT  techniques.  The  arguments  for  MADA  versus  MAUT  are  extensive  and  date 
back  to  the  1950 's.  The  arguments  against  MADA  by  utility  theorists  have  not  “ killed ” this 
approach.  Indeed,  MADA  is  today  applied  extensively  by  Fortune  500  companies  and  national 
governments  worldwide.  Also,  since  MADA  is  not  a MAUT  approach,  it  has  different  conditions. 
In  particular,  congruence  between  the  weighting  step  and  the  categoiy  indicators  is  not 
required. 

- The  two  provided  default  weighting  schemes  from  SAB  and  Harvard  both  refer  to  the 
importance  of  environmental  problems  if  the  total  threat  to  the  U.S.  is  considered.  This  threat  is 
primarily  caused  by  U.S.  emissions  and  extractions,  plus  some  imports  minus  exports.  If  these 
weights  should  be  applied  to  the  normalized  scores  of  BEES  this  means  that  you  have  to  assume 
that  first  the  building  material  sector  causes  exact  proportional  shares  of  environmental  problems 
as  observed  on  a total  economy  level,  and  second  that  the  worst  indicators  of  the  actually- 
compared  building  components  are  again  proportional  to  the  average  emissions  and  resource 
extractions  of  the  building  industry.  If  we  would  believe  this  to  be  true,  then  the  BEES  analysis 
could  be  dramatically  simplified  (the  analysis  of  one  impact  category  would  be  sufficient).  The 
use  of  the  SAB  and  Harvard  weights  also  implies  that  they  correspond  with  the  implied  meaning 
of  the  decision  at  hand.  Building  product  decisions  will  change  the  marginal  burden  to  the 
environment.  Therefore,  the  weights  should  reflect  the  importance  of  increases  or  decreases  of 
the  different  effects,  rather  than  their  present  total  magnitude. 

NIST  Response:  This  is  a specious  argument.  In  BEES,  product  flows  contributing  to 
environmental  impacts  are  penalized  in  proportion  to  the  severity > of  the  impacts  from  a U.S. 
perspective.  NIST  is  simply  assuming  a constant  marginal  damage  function. 


-The  offer  to  weight  normalized  scores  equally  means  in  practice  that  the  exchange  rate  between 
kg  C02-equivalents  and  kg  P04-equivalents  depends  on  whether  I compare  wall  insulations  or 
flooring  materials  or  whether  the  comparison  of  flooring  materials  includes  linoleum  or  not. 
Such  a dependence  on  the  number  and  nature  of  alternatives  is  considered  to  be  a flaw  of  a 
procedure. 

NIST  Response:  See  Major  Responses  #7  and  #2  to  MADA  Critique  above. 


-In  the  fourth  weighting  alternative  - the  setting  of  user-defined  weights  - congruence  is  not 
possible  because  the  user  does  not  know  what  s/he  is  weighting.  The  magnitude  of  the  weighted 
scores  is  not  provided. 

NIST  Response:  See  Major  Response  #2  to  MADA  Critique  above. 
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- Von  Neumann  and  Morgenstern  (1944)  introduced  some  conditions  for  expected  utility  theory 
and  included  the  condition  (among  others)  of  transitivity.  As  you  admit  on  page  35  ‘rank 
reversal’  occurs  in  the  chosen  method.  Therefore,  one  of  the  most  basic  conditions  for  the 
expected  utility  theory  is  violated  and  the  approach  flawed. 

NIST  Response:  See  Major  Response  #2  to  MAD  A Critique  above.  Transitivity ? is  not  required  by 
MAD  A.  Indeed , while  utility > theorists  argue  that  people ’s  decisionmaking  must  be  absolutely 
consistent , MADA  theorists  argue  that  absolute  consistency  is  neither  necessary  nor  desirable. 


- Based  on  the  above,  the  BEES  normalization  procedure  has  to  be  changed  to  establish 
congruence  between  the  weights  and  the  category  indicators.  The  most  common  approach  uses 
normalization  values  that  reflect  the  contribution  of  a certain  region  during  a defined  time  to  the 
problems  (environmental  impacts)  at  hand.  However,  the  congruence  criterion  requires  that  the 
actually-used  reference  system  be  reflected,  which  may  require  different  normalization  sets  for 
the  different  suggested  weighting  sets. 

- Jane  Bare  (U.S.  EPA)  may  be  able  to  provide  U.S.  data  for  the  normalization  step  that  are 
being  developed  for  her  TRACI  project. 

NIST  Response:  See  Major  Responses  #1  and  #2  to  MADA  Critique  above. 

- Hofstetter  et  al.  (2000)  provide  a graphical  method  that  helps  to  support  decisions  when  trade- 
offs have  to  be  made  with  a minimum  of  weighting  information.  (In  some  cases  the 
environmental ly-best  alternative  is  also  the  one  with  the  lowest  LCC.  In  such  cases,  the  user 
should  know  that  the  weighting  of  costs  and  environmental  burden  have  no  relevance  to  the 
ranking  outcome). 

- The  transformation  of  the  SAB  and  Harvard  verbal  importance  rankings  to  weights  on  cardinal 
scales  includes  a lot  of  assumptions  that  are  not  made  explicit.  The  AHP  process  is  controversial 
and  the  interpretation  in  this  case  arbitrary.  Why  didn’t  you  choose  factors  of  3 and  7?  It  has  to 
be  assumed  that  members  of  both  panels  would  discourage  the  use  of  these  weights. 

NIST  Response:  The  factors  were  chosen  through  an  iterative  process  based  on  NIST's 
background  and  experience  in  applying  the  AHP  pairwise  comparison  process.  The  pairwise 
comparison  process  is  intended  for  translation  of  verbal  importance  rankings  to  numerical 
importance  weights,  so  panel  member  support,  while  desirable,  is  not  a prerequisite.  For  BEES 
3.0,  NIST  will  expand  its  discussion  of  its  transformation  of  weights  to  cardinal  scales.  Also  note 
that  BEES  permits  users  to  adjust  any  of  the  weights. 


- The  rounding  of  values  used  in  Tables  2.13  and  2.15,  and  for  the  equal  weights,  is  appropriate 
with  regard  to  its  uncertainty  but  is  very  confusing  because  the  equal  judgments  on  some 
problems  are  not  evident. 

NIST  Response:  For  BEES  3.0,  NIST  will  clarify  the  rounding  of  weights. 
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8.  Data  and  Data  Sources 


-A  strong  feature  of  the  tool  is  that  it  includes  a reasonably  large  set  of  LCI  data.  This  is  of 
course  the  (expensive)  heart  of  the  tool.  The  quality  of  the  data  was  not  reviewed  due  to  the 
lacking  documentation. 


- The  presented  alternatives  for  individual  building  elements  may  cover  the  most  frequently-used 
products,  but  do  not  include  eco-leaders.  This  means  that  additional  alternatives,  like  highly- 
insulated  walls  (R-value  higher  than  R-25  or  R-30)  and  roofs,  should  be  included.  Also,  unpaved 
parkways  that  allow  retention  of  water  should  be  added  as  viable  alternatives  (these  are  small 
concrete  blocks  that  protect  soil  from  compression  but  allow  for  vegetation).  Only  the  addition  of 
these  eco-leaders  may  motivate  designers  to  go  for  a better  alternative.  Eco-leaders  should 
always  appear  as  benchmarks  in  comparisons,  i.e..  even  if  not  selected. 

Major  Response  to  Data  Suggestions 

NIST  Response:  To  date,  BEES  products  have  been  selected  on  the  basis  of  sponsor  interest, 
intriguing  product  comparisons,  and  extensive  building  element  coverage.  It  is  not  possible  to 
identify  eco-leaders  a priori  because  weights  can  change  and  tradeoffs  among  multiple  impacts 
are  not  always  as  expected.  Presumably,  the  BEES  Please  program  will  encourage  perceived 
eco-leading  manufacturers,  including  those  marketing  the  products  cited  above,  to  submit  then- 
products  for  inclusion  in  BEES.  Further,  the  incorporation  ofU.S.  normalization  data  into  BEES 
3. 0 will  offer  more  consistent  benchmark  comparisons,  because  eco-leaders  will  change  based 
on  user-defined  weights. 


R-l  1 through  R-15  wall  insulation  do  not  really  motivate  environmentally-relevant 
improvements.  Such  building  practice  means  that  the  buildings  are  using  far  too  much  energy  in 
the  use  phase  for  another  50  years  to  100  years!  What  are  the  assumptions  on  the  remaining  wall 
structure  when  you  calculate  the  difference  in  heating  use?  By  setting  future  costs  due  to  energy 
supply  to  zero  for  the  best  insulation  material,  you  underestimate  the  effective  LCC  and  give 
little  incentive  to  reduce  total  LCC. 

NIST  Response:  See  Major  Response  to  Data  Suggestions  above.  BEES  assumes  identical 
remaining  wall  structures  when  evaluating  use  energy  differences  among  insulation  alternatives. 
However,  as  mentioned  in  the  Future  Directions  section  of  the  BEES  documentation,  in  future 
versions  of  BEES,  NIST  intends  to  begin  combining  building  products  to  permit  comparative 
analyses  of  entire  building  components  and  assemblies,  a necessary’  first  step  permitting 
analyses  of  different  wall  structures  in  combination  with  insulation  alternatives.  The  best 
insulator ’s  use  energy  is  set  to  zero  in  line  with  BEES  ’ design  as  a product  selection  tool. 
Otherwise,  BEES  would  be  overestimating  the  effect  on  use  energy  of  insulation  product  choices 
by  penalizing  insulation  products  for  the  building's  entire  heating  load. 

- It  is  mentioned  on  page  8 of  the  user  manual  that  the  data  have  been  checked  by  industry.  Is 
this  review  documented?  Based  on  which  information  were  the  data  checked?  This  could  be 
helpful  when  the  whole  database  is  reviewed  in  the  future. 
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NIST  Response:  Industry  reviewers  were  all  sent  Beta  versions  of  BEES  2.0  and  its 
documentation.  NIST  resolved  and  incorporated  valid  industry  comments,  involving  both  LCI 
assumptions  and  models,  and  in  some  cases  involving  face-to-face  meetings,  before  publishing 
BEES  2.0.  For  confidentiality  reasons,  and  to  encourage  future  industry  participation  in  the 
BEES  project,  it  is  not  possible  to  publish  written  industry  comments.  However,  as  noted  in  its 
BEES  Please  announcement,  NIST  plans  to  automatically  update  its  LCI  data  periodically,  and 
invites  industry  to  submit  earlier  updates  as  technology’  changes  warrant. 


- Cement  production  is  known  for  high  heavy  metal  emissions.  Are  they  reported?  The 
production  of  nylon  is  supposed  to  have  significant  N2O  emissions.  Are  they  included? 

NIST  Response:  Heavy  metals  are  reported  and  evaluated  in  the  ecological  and  human  toxicity > 
assessments  included  for  parking  lot  paving  alternatives.  The  other  concrete  alternatives 
evaluate  only  6 impacts  and  so  do  not  include  these  impacts.  Yes,  nitrous  oxide  emissions  from 
nylon  production  are  reported  and  included  in  the  BEES  global  warming  assessment. 

- The  information  provided  on  the  LCI  data  is  insufficient  to  check  whether  the  analysis  is  in 
accordance  with  ISO  14041.  Therefore,  the  transparency  criterion  is  violated. 

Chair  Note:  It  has  already  been  stated  at  the  beginning  of  this  review  document  that  the 
reviewers  did  not  inspect  data;  the  review  is  limited  to  the  modeling  and  presentation  of  results 

Major  Response  to  Transparency  Comment 

NIST  Response:  Reliable,  transparent  U.S.  LCI  data  are,  at  present,  only  available  from 
commercial  tools.  BEES  does  not  publish  fully  transparent  data  because  none  are  publicly 
available  and  NIST  does  not  intend  to  compete  with  private  sector  tools  by  placing  these  data  in 
the  public  domain.  Rather,  NIST  contracts  with  Environmental  Strategies  and  Solutions  and 
PricewaterhouseCoopers  ( PwCj/Ecobalance,  a world-class  LCI  data  developer,  for  aggregated 
LCI  data,  thus  protecting  PwC ’s  intellectual  property > as  well  as  manufacturers ' trade  secrets. 
The  PwC  commercial  LCI  data  product,  DEAM™,  from  which  much  of  the  BEES  “ backbone  ” 
data  are  taken,  strictly  follows  the  ISO  14041  transparency  criterion.  Users  seeking  full 
transparency  may  purchase  DEAM™ for  complete  documentation  on  its  upstream  data  sources 
and  calculation  procedures.  NIST  seeks  to  provide  a public  sendee  by  making  available  at  no 
charge  LCI  data  that  are  transparent  in  an  aggregated  sense.  However,  once  U.S.  backbone  LCI 
data  become  available  in  the  public  domain,  NIST  will  readily  incorporate  these  fully 
transparent  data  into  BEES.  Finally,  NIST ’s  transparency  concern  is  primarily  focused  on 
permitting  the  user  to  reproduce  all  the  BEES  scores  by  providing  the  algorithms  and  all  the 
aggregated  data  that  are  used  in  them. 

- Cut-off  rules:  Figure  2.1  provides  a good  overview  of  the  chosen  cut-off  criteria.  The  chosen 
criteria  do  not  address  the  low-weight,  low-energy,  low-cost  inventory  flows  that  have  a high 
environmental  contribution.  Processes  with  high  (fugitive)  process-specific  emissions  tend  in  this 
framework  to  be  excluded  (a  violation  of  cut-off  criteria  in  ISO  14041) 

NIST  Response:  The  BEES  contracting  team  does  not  agree  with  this  strict  interpretation  of  the 
ISO  cut-off  rules,  and  thus  does  not  believe  the  BEES  cut-off  rules  violate  ISO  14041.  Section 
5.2.4  (Criteria  for  initial  inclusion  of  inputs  and  outputs ) in  ISO  14041  seems  to  generally 
support  the  cut-off  rules  described  under  Section  2.1.1,  Goal  and  Scope  Definition,  of  the  BEES 
Technical  Manual.  ISO  14041  states: 
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During  the  scope  definition,  the  initial  set  of  inputs  and  outputs  are  selected  for  the  inventory.  This 
process  recognizes  that  it  often  not  practical  to  model  every  input  and  output  into  the  product  system.  It  is 
an  iterative  process  to  identify  the  inputs  and  outputs  which  should  be  traced  to  nature,  i.  e.  which  unit 
processes  producing  the  inputs  or  which  unit  processes  receiving  the  outputs,  should  be  included  in  the 
product  system  under  study.  The  initial  identification  will  typically  be  made  using  available  data,  and 
inputs  and  outputs  should  be  more  fully  identified  when  additional  data  are  collected  during  the  course  of 
the  study,  and  subject  to  a sensitivity  analysis.  ...  The  criteria  and  assumptions  on  which  they  are 

established  shall  be  clearly  described.  There  are  several  criteria  that  are  used  in  LCA  practice  to 

decide  which  inputs  will  be  studied,  including  1)  mass,  2)  energy’,  and  3)  environmental  relevance.  Making 
the  initial  identification  of  inputs  based  on  mass  contribution  alone  may  result  in  important  inputs  being 
omitted  from  the  study.  Accordingly,  energ y and  environmental  relevance  should  also  be  used  as  criteria 
in  this  process.  " 

In  BEES,  energy’  and  cost  (a  proxy  for  environmental  relevance)  criteria  are  used  in  addition  to 
the  mass  criterion.  Finally,  the  cut-off  rules  used  in  BEES  cover  the  majority ; of  flows;  the 
example  raised  by  the  peer  reviewer  is  more  the  exception  than  the  rule.  With  additional  funding, 
NIST  could  refine  its  cut-off  rules  to  include  sensitivity > analyses  to  determine  which  flows  are  the 
most  environmentally  relevant. 

- One  of  the  most  significant  limitations  of  the  BEES  Program  is  that  it  does  not  incorporate  a 
number  of  commonly-specified  generic  building  products  that  collectively  offer  numerous 
opportunities  to  incorporate  green  materials.  Porous  paving  products,  recycled-content  aggregate 
base  and  sub-base,  recycled  asphalt  concrete,  rubberized  asphalt  paving,  exterior  cladding, 
curtain  wall  systems,  re-locatable  interior  wall  systems,  toilet  partitions,  plastic  lumber, 
acoustical  ceiling  tile,  standard  gypsum  board,  synthetic  gypsum  board,  high  impact  gypsum 
board,  finished  decorative  wall  panels,  and  structural  engineered  wall  systems  are  among  the 
generic  material  and  product  classifications  that  are  not  addressed  in  BEES  2.0.  The  reviewer 
notes  that  the  National  Institute  of  Standards  and  Technology  (NIST)  has  already  recognized  this 
limitation  and  will  address  the  need  to  expand  the  product  selections  in  future  work. 

NIST  Response:  Agreed.  Collecting  BEES  LCI  data  is  a very  expensive  undertaking  for  NIST. 
NIST  asks  for  patience  as  it  continues  to  incorporate  these  important  generic  products  into 
future  versions  of  BEES. 


- Another  important  limitation  of  the  BEES  software  is  that  the  results  mostly  reflect  industry 
averages  compiled  for  generic  product  types.  Professional  designers  and  other  decision-makers 
may  consider  BEES  a useful  tool  for  evaluating  very  basic  material  selections  made  during  the 
conceptual  design  and  design  development  phases  of  a project.  The  software  can  identify 
relevant  cost  and  environmental  distinctions  between  these  generic  material/product 
classifications. 

Major  Response  to  Industry-Average  Data  Limitation 

NIST  Response:  To  address  this  limitation,  NIST  has  already  launched  its  BEES  Please  program 
soliciting  brand-specific  LCI  data  from  manufacturers  for  incorporation  into  BEES  3.0. 


- Design  professionals  often  need  product-specific  information  during  the  bidding  phase  of  the 
project  to  educate  the  building  team  and  building  owners  about  green  material  choices  and  to 
process  substitution  requests  for  green  building  materials.  This  is  important  because  significant 
differences  in  both  cost  and  environmental  performance  characteristics  often  exist  between  many 
individual  products  with  the  same  application.  And  it  is  clear  that  environmental  differentiation 
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among  individual  products  is  becoming  a trend.  This  trend  is  market-driven  and  measured  by  the 
growing  number  of  companies  that  produce  green  products  to  meet  the  demand  for  green 
building  and  green  construction.  It  is  noted  that  the  BEES  manual  addresses  the  limitations 
related  to  the  lack  of  individual  product  selections  and  has  already  identified  the  need  to  “add 
more  products  to  currently  covered  elements.”  To  illustrate  the  point  made  above,  an  interior 
designer  evaluating  flooring  options  might  use  BEES  to  compare  conventional  vinyl  composition 
tile  with  ceramic  tile.  However,  if  the  interior  designer  selects  the  VCT  flooring  option,  BEES 
does  not  permit  the  user  to  evaluate  product  choices  that  fall  within  the  “VCT  classification”  or 
reasonable  alternatives  to  VCT  based  on  the  performance,  cost,  and  application  requirements. 

For  instance,  the  user  cannot  compare  standard  VCT  with  an  alternate  resilient  flooring  product 
made  from  a chlorine-free,  mineral-filled  ethylene  co-polymer  backing  (made  by  Amtico).  The 
latter  product  has  relatively  low  VOC  emissions  (4  week  rating  is  < 10  ug/irf/hr)  and  is 
reportedly  twice  as  resistant  to  abrasion  than  VCT.  The  indications  are  that  the  latter  product  has 
significantly  better  environmental  performance  than  conventional  VCT.  By  the  same  token,  the 
development  of  detailed  product  selections  in  the  BEES  Program  would  be  very  beneficial  for 
differentiating  cost  and  environmental  performance  within  the  generic  “nylon  commercial  carpet 
classification.”  There  are  only  four  general  selections  available  in  BEES  for  nylon  commercial 
carpet:  a)  nylon  carpet  tile  with  traditional  glue,  b)  nylon  carpet  tile  with  low-VOC  glue,  c)  nylon 
broadloom  carpet  with  traditional  glue,  and  d)  nylon  broadloom  carpet  with  low-VOC  glue. 

More  detailed  product  selections  are  warranted  to  incorporate  critical  performance  criteria  for 
indoor  air  quality,  material  re-use,  recycled  content,  carpet  recycling  and  product  design  life.  To 
illustrate  this  point,  the  reviewer  proposes  more  detailed  nylon  carpet  specifications,  which  are 
listed  below:  (1)  For  both  broadloom  and  tile,  nylon  carpet  with  integral  backing,  installed  with 
low-VOC  glue,  certification  that  product  meets  CRI  emission  guidelines  for  indoor  air  quality, 
certification  that  recycled  content  is  at  least  45  % by  total  product  weight,  warranty  for  surface 
wear  not  more  than  1 0 % by  mass  fraction  in  1 5 years,  and  certification  that  manufacturer  has 
carpet  reclamation  program  in  place  to  provide  for  recycling  at  end  of  its  useful  life  and  (2)  For 
both  broadloom  and  tile,  refurbished  nylon  carpet  with  integral  backing,  installed  with  low-VOC 
glue,  certification  that  product  meets  CRI  emissions  guidelines  for  indoor  air  quality,  warranty 
for  surface  wear  not  more  than  10  % by  mass  fraction  in  10  years,  and  certification  that 
manufacturer  has  carpet  reclamation  program  in  place  to  provide  for  recycling  at  end  of  its  useful 
life. 

NIST  Response:  See  Major  Response  to  Industry-Average  Data  Limitation  above. 

- The  BEES  Program  includes  a number  of  product  choices  for  reinforced  concrete  applications, 
including  foundations,  basement  walls,  concrete  beams,  and  columns.  For  most  of  these  building 
elements  made  from  concrete,  the  primary  selections  are  for  concrete  materials  having  varying 
levels  of  fly  ash  content  by  weight  of  Portland  cement.  In  the  opinion  of  the  reviewer,  the  BEES 
Program  would  benefit  from  additional  selections  for  structural  and  non-structural  concrete 
building  elements,  in  particular  pre-cast  concrete,  cast-in-place  concrete,  autoclaved  aerated 
concrete,  and  concrete  containing  recycled  aggregates.  For  example,  a comparison  of  pre-cast 
versus  cast-in-place  products  (e.g.,  for  support  columns,  structural  beams,  concrete  pipe,  and 
other  products  in  the  same  generic  classification)  would  disclose  some  interesting  environmental 
performance  tradeoffs,  in  particular  differences  resulting  from  the  need  to  construct  temporary 
formwork  (required  for  cast-in-place),  generation  of  construction  waste  (less  in  pre-cast), 
transportation  energy  impacts  (probably  higher  in  pre-cast),  weight-to-strength  ratio  (some  pre- 
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cast  products  are  pre-stressed)  and  potential  fly  ash  content  (use  of  high  volume  fly  ash  content  is 
probably  more  suitable  in  cast-in-place  applications).  The  BEES  approach  would  be  useful  in 
helping  design  professionals  understand  how  these  differences  coalesce  in  terms  of  overall 
environmental  performance  and  cost. 

NIST  Response:  Agreed.  Collecting  BEES  LCI  data  is  a very  expensive  undertaking  for  NIST. 
NIST  asks  its  BEES  users  for  patience  as  it  continues  to  incorporate  these  important  generic 
products  into  future  versions  o f the  tool. 

- Environmental  assessments  for  specific  environmental  impacts  can  vary  significantly 
according  to  regional  conditions  and  circumstances.  However,  as  stated  in  the  BEES  Manual, 

“the  goal  of  BEES  LCA  assessment  is  to  generate  U.S.  average  results.”  In  the  opinion  of  the 
reviewer,  this  is  not  a desirable  simplification  because  it  limits  the  usefulness  the  BEES  results. 
(The  goal  is  probably  due  to  data  limitations  and  the  complexity  of  programming  the  LCA 
models).  The  reviewer  notes  that  the  BEES  manual  identifies  a goal  to  “refine  all  data  to  permit 
U.S.  region-specific  BEES  analyses”  as  a proposed  data  enhancement  for  future  development. 
NIST  Response:  Yes,  to  date,  data  availability  and  collection  cost  limitations  have  been  the 
basis  for  the  U.S.  scope.  However,  as  noted,  NIST  intends  to  refine  all  data  to  permit  U.S. 
region-specific  analyses  in  future  versions  of  BEES. 


- There  is  an  apparent  discrepancy  between  the  BEES  results  and  actual  California-specific 
project  cost  information  for  incorporating  fly  ash  in  concrete  for  various  building  elements,  such 
as  foundations,  paving,  and  basement  walls.  For  example,  the  BEES  results  indicate  that 
foundations  made  from  fly  ash  are  significantly  less  expensive  on  a first  cost  basis  than  concrete 
foundations  containing  no  fly  ash.  Unfortunately,  the  BEES  results  may  not  reflect  the  true  cost 
of  high- volume  fly  ash  foundations  (and  probably  other  concrete  building  elements)  constructed 
in  California.  This  observation  is  based  on  case  study  information  known  to  the  reviewer.  For 
example,  a 61 16  m3  (8,000  yd3),  40  % fly  ash  slab  foundation  poured  in  Sacramento  costs  $2.63/ 
m3  ($2/yd3)  more  than  the  projected  cost  for  the  same  foundation  without  fly  ash.  The  exact 
reasons  for  the  California  cost  discrepancy  are  unclear,  but  may  be  partly  attributable  to  the  fact 
that  fly  ash  from  coal-fired  power  plants  is  not  a locally  available  resource  in  California.  This 
discrepancy  suggests  that  the  BEES  Program  may  benefit  from  additional  cost  research  in  this 
area.  If  a significant  regional  cost  anomaly  for  fly  ash  is  documented,  the  problem  could  be 
corrected  by  creating  a variable  in  the  BEES  program  to  allow  the  results  to  reflect  regional  cost 
differences  in  this  classification.  The  reviewer  realizes  that  the  current  goal  of  BEES  is  to 
establish  a scope  based  on  U.S.  average  results. 

NIST  Response:  Yes,  local  availability’  most  likely  drives  relative  first  costs  for  fly-ash-content 
versus  100  % cement-  content  concretes.  When  refining  environmental  data  to  permit  U.S 
region-specific  analyses  as  noted  above,  NIST  will  also  refine  economic  data  to  account  for 
regional  price  differences . 


- Over  90  % of  the  data  used  in  the  assessments  come  directly  from  industry  sources.  Since  a 
majority  of  LCA  databases  are  European,  how  was  the  conversion  to  U.S.  operations  done? 
NIST  Response:  For  about  99  % of  the  BEES  LCI  data,  conversion  from  European  LCI 
databases  was  not  necessaiy.  From  the  very  beginning  of  the  BEES  project,  the  NIST  LCI  data 
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contracting  team , Environmental  Strategies  & Solutions  and 

PricewaterhouseCoopers/Ecobalance , has  contacted  appropriate  U.S.  industry  representatives 
for  each  and  every  product  incorporated  into  BEES.  Based  on  input  from  these  U.S.  contacts,  the 
team  conducts  ne it'  research  for  each  product  permitting  development  of  gate-to-grave  LC1  data 
sets  based  primarily  on  U.S.  industry  data.  In  addition,  the  expert  contracting  team  applies 
Ecoba/ance ’s  commercial  U.S.  LCI  data  tool,  DEAM™,  to  develop  much,  if  not  all,  of  the  cradle- 
to-gate  LCI  data  that  are  then  combined  with  the  gate-to-grave  industry  data  to  derive  complete, 
cradle-to-grave  BEES  LCI  data  sets.  Ninety  percent  of  the  crad/e-to-gate  DEAM™  data  come 
directly  from  industry  sources.  Industry  representatives  are  again  contacted,  this  time  by  NIST, 
once  LCI  data  for  their  products  have  been  incorporated  into  BEES,  for  a second  opportunity’  to 
provide  input  before  LCI  data  for  their  products  are  published  in  BEES.  While  this  is  an 
expensive  route  to  take,  it  is  the  only  way  to  ensure  data  quality’. 
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9.  User  Support 


- If  someone  needs  to  ask  the  BEES  developers  a question  (or  supply  a good  idea  for  a future 
version),  it  would  help  if  contact  info  (email,  phone)  were  placed  either  in  the  “About...” 
window,  or  in  a separate  “Technical  Support”  - type  window. 

NIST  Response:  NIST  will  add  contact  information  to  its  BEES  3. 0 documentation. 

After  1 month  of  ordering  the  material,  it  has  still  not  arrived. 

NIST  Response:  At  the  time  of  the  peer  review,  there  was  an  extensive  delay  in  shipping  the 
printed  materials.  It  has  since  been  corrected.  Our  apologies. 
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10.  User  Manual 


- The  BEES  Manual  (page  15)  does  not  state  which  specific  solid  waste  fractions  (generated 
from  various  stages  and  segments  of  the  building  industry)  are  considered  “non-recyclable.”  In 
fact,  a material  that  is  deemed  “non-recyclable”  in  one  region  (perhaps  due  to  lack  of  processors, 
low  tipping  fees,  no  mandatory  recycling  laws)  may  be  considered  “recyclable’'  in  another  region 
with  more  favorable  conditions.  This  information  needs  to  be  clarified  in  the  BEES  manual. 
NIST  Response:  In  future  versions  of  BEES , NIST  will  add  a table  specifying  the  U.S.  average 
non-recyclable  fractions  used , and  may  parameterize  this  factor  to  accommodate  regional 
variations. 

- Throughout  the  manual  comments  are  made  about  the  limitations  of  the  model.  These  need  to 
be  strong  statements.  Even  though  the  limitations  are  mentioned,  will  the  user  consider  them  in 
their  decision  process?  The  model  is  easy  to  use,  just  point  and  click.  Will  the  user  read  the 
manual  or  just  go  to  the  tutorial  and  learn  how  to  use  the  tool  and  start  their  assessments? 

NIST  Response:  Limitations  are  also  clearly  given  in  the  software  to  inform  users  not  inclined  to 
read  the  manual:  a maximized  help  window  with  the  Limitations  discussion  clearly  marked 
automatically  opens  each  and  even’  time  the  user  opens  BEES.  NIST  is  open  to  suggestions  as  to 
how  to  further  address  this  issue. 

- The  comment  made  above  also  applies  to  the  assumptions  made  in  each  LCA.  Will  the  user 
determine  if  their  process  is  similar  to  the  assumptions  made  and  then  continue  with  the  analysis, 
or  will  the  user  just  start  using  the  model  and  not  consider  the  assumptions? 

NIST  Response:  To  address  this  concern,  NIST  will  add  a “ Product  Details  ” button  on  the  BEES 
3.0  Alternatives  window  that,  when  pressed,  documents  assumptions  for  each  product.  In 
addition,  the  BEES  Please  program  for  incorporating  brand-specific  data  will  help  by 
accounting  for  variation  among  products  within  an  industry. 
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11.  Other  Recommendations 


All  suggestions  that  would  lead  to  a broadening  of  the  user  group  should  lead  to  separate 
software  tools  rather  than  a “one  size  fits  all”  tool  that  is  as  difficult  to  use  as  the  commercial 
LCA  software  on  the  market. 

NIST  Response:  Agreed. 
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